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(54) Abstract Title 

Statistically monitoring a communications system against a variable threshold 



(57) A communications system is monitored using statistics derived from operating parameters, conditions 
and event data within the communications system. The statistics are collected from the communications 
system and statistically analysed with reference to a threshold. The threshold is varied and in one 
embodiment this variation is represented as a function of the sample size. Different functions may be given to 
different ranges of sample sizes so that for example the threshold in a first range 410 may take the form of a 
linear equation, whilst in the second range 420 it may be constant. The communications system may be 
cellular and the cells may be grouped into respective groups having different thresholds. The statistical data 
may relate to the number of calls requested, setup, in progress and dropped etc. 
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least one drawing originally filed was informal and the prim reproduced here is taken from a later filed formal copy. 
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MONITORT NG A COMMT TNIC ATTONS SYSTEM 
Field of the Invention 



The present invention relates to a method of monitoring a communications 
system using statistics derived from the communications system. The present 
invention also relates to an apparatus for monitoring a communications system 
using statistics derived from the communications system. The present invention 
is particularly suitable for a cellular communications system, and for example is 
applicable, but not limited to, a GSM system. 



Background of the Invention 



In communications systems, large amounts of data are produced with respect to 
a multitude of operating parameters, conditions and events in the system. 
Examples include data related to number of calls requested, number of calls 
successfully set-up, number of calls in progress, number of dropped calls, and so 
on. It is useful to collect some or all of this data for analysis so as to provide 
information about the system and its performance. The data that is collected and 
analysed represents statistics derived from the communications system. The 
statistics can in theory be all of the available data, but usually in practice is some 
of the available data, most usually selected by some form of statistical sampling 
procedure. 

In order to monitor the performance of some aspect of a communications 
system, the statistics are analysed with reference to a threshold. Then if the 
threshold is exceeded, this fact can be indicated and attention given to the 
situation. Such indication can involve human operators or can be entirely 
automatic. The monitoring of the communications system can be carried out in 
real time, wherein statistics are collected on an ongoing basis and compared to 
the threshold continually, or at least such comparison is repeated at regular 
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intervals, say, or the monitoring can be carried out off-line after a discrete 
amount of statistics have been collected. 

One type of communications system is a cellular communications system. In a 
cellular communications system, the area over which service is provided is 
divided into a number of smaller areas called cells. Typically each cell is served 
from a base transceiver station (BTS) which has a corresponding antenna or 
antennas for transmission to and reception from a user station, normally a 
mobile station. Presently established cellular radio communications systems 
include GSM systems (Global System for Mobile Communications). Monitoring 
of cellular communications system is particularly advantageous because the 
mobility of user stations, and ongoing interactions between different cells, 
produce ever changing performance situations in addition to those 
characteristics, such as faults and capacity levels, which are also found in non- 
cellular systems. 

However, problems arise in known methods employing thresholds, particularly 
in methods using at least some degree of automation. Such thresholds can be 
clumsy in their effect, and produce results that are not always appropriate. 

One disadvantage arising from the use of a threshold is that when sample sizes 
are small, assessment with respect to a threshold can produce statistically 
distorted results, in that the naturally occurring statistical variations in a small 
sample size can give a false crossing of a threshold even when the system under 
consideration is not fundamentally crossing the threshold. In known systems 
this problem is countered by excluding too small sample sizes from analysis in 
the monitoring process, or alternatively by increasing the sample size. 
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Another disadvantage arising from the use of a threshold is that a threshold that 
is appropriate in one area of the system may not be appropriate in another part 
of the system. 

Summary of the Invention 

The present invention addresses some or all of the above disadvantages, by 
virtue of providing that the threshold is varied. 

According to one aspect of the present invention, there is provided a method of 
monitoring a communications system using statistics derived from said 
communications system, as claimed in claim 1. 

According to another aspect of the invention, there is provided an apparatus for 
monitoring a communications system using statistics derived from said 
communications system, as claimed in claim 11. 

Further aspects of the invention are as claimed in the dependent claims. 

The present invention tends to provide a measure of improvement to the use of 
a threshold, with ensuing advantages. By varying the threshold, disadvantages 
arising from the fixed nature of known thresholds are overcome. 

More particularly, when according to one aspect of the present invention the 
threshold is varied as a function of a sample size of said statistics derived from 
said communications system, then the effect is to make advantageous use of 
data that would need to be discarded in known methods, whilst avoiding or 
reducing disadvantages that arise from the stronger effect of statistical variation 
at lower sample size. This is of powerful benefit in the case of real time 
monitoring since the provision for monitoring based on smaller sample size 
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allows quicker detection of faults or other system changes, disturbances or 
phenomena, compared to known methods. 

Furthermore, when according to another aspect of the present invention the 
5 method comprises the step of respectively allocating cells of said cellular 
communications system into a plurality of groups according to values of 
characteristics of said cells, and wherein variation of said threshold comprises 
employing a respective group threshold value for each respective one of said 
plurality of groups, then the effect is to advantageously provide different 
10 threshold levels in different geographical areas of the system. This is beneficial 
since, for example, it is advantageous if different call set-up failure rates are 
indicated in an urban area compared to a rural area. 

Yet further, in a cellular communications system employing many cells, which 
15 cells moreover may often be changed in their nature due to the insertion of 
further cells, the setting-up of a monitoring plan involving different thresholds 
for different cells would be very time consuming and prone to errors. This 
further disadvantage is alleviated when, according to yet another aspect of the 
present invention, said cells are allocated into said plurality of groups 
20 automatically and wherein said values of characteristics of said cells comprise 
statistics collected automatically from said cellular communications system. 

Additional specific advantages are apparent from the following description and 
figures. 

25 

Brief Description of the Drawing s 



30 



FIG. 1 is an illustration of a cellular communications system which is in 
accordance with the present invention. 
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FIG. 2 is a process flow chart of an embodiment of the present invention. 

FIG. 3 is a schematic illustration of threshold variation as a function of a sample 
size in an embodiment of the present invention. 

FIG. 4 is a schematic illustration of threshold variation as a function of a sample 
size in another embodiment of the present invention. 

FIG. 5 is a process flow chart of another embodiment of the present invention. 

Description of Embodime nts of the Invention 

The embodiments hereinafter described relate to a cellular communications 
system which is a GSM system, although it will be appreciated that the 
invention is not limited to such a system and can equally be used in non-cellular 
communications systems, including public switched telephone networks, 
private network radio systems, and so on. The invention can also be used in 
other cellular communications systems, including other time division multiple 
access (TDMA) systems, code division multiple access (CDMA) systems, i 
combined TDMA -CDMA systems. 



le 

or 



HG. 1 illustrates part of a cellular communications system 100 including a 
mobile services switching centre (MSC) 110 to which is coupled a public 
switched telephone network (PSTN) 120 and an operations and maintenance 
centre (OMC) 130. Further coupled to MSC 110 is base station controller (BSC) 
140. Yet further, base transceiver stations (BTS) 150, 160, 170 and 180 are coupled 
to MSC 110 through BSC 140. Each BTS 150, 160, 170 and 180 provides a 
respective coverage area, each known as a cell 152, 162, 172, 182 of the cellular 
communications system. By convention, each cell 152, 162, 172, 182 is shown in 
FIG. 1 in the form of a hexagon, but in real systems the shape of each cell will 
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depend on a number of details such as transmitter power, geographical details, 
and so on. 

Two user stations, in the form of mobile stations, are shown in each cell - mobile 
stations 154 and 156 in cell 152, mobile stations 164 and 166 in cell 162, mobile 
stations 174 and 176 in cell 172, and mobile stations 184 and 186 in cell 182. 

The method of monitoring the communications system using statistics derived 
from said communications system according to a first embodiment is shown in 
process flow chart 200 of FIG. 2. Referring to FIG. 2, function box 210 shows the 
step of collecting statistics from said communications system. In the present 
embodiment the statistics are derived from the communications system by 
virtue of OMC 130 retrieving data corresponding to attempted call set-ups by 
and to the above mentioned mobile stations. Such data is originally produced in 
any or all of BTS 150, BTS 160, BIS 170, BTS 180, BSC 140 and MSG 110, and is 
transmitted to MSC 110 where it is collected by OMC 130. OMC 130 further 
carries out statistical sampling of the data, in the present embodiment 
determining a rate of success of call set-up, thus collecting the relevant statistics 
which in the present embodiment thereby comprise the rate of success of call 
set-up in cells 152, 162, 172 and 182. 

Other types of data that could be employed include drop call rate, handover 
success rate, traffic channel congestion, signalling channel congestion, paging 
volume and call volume. 

The next step is that of statistically analysing said statistics with reference to a 
threshold, as shown at function box 220 of FIG. 2. In the present embodiment, let 
us assume the system operator considers a call set-up rate of 90% success, i.e. 
10% rate of failure, to be acceptable. It is noted that many commercial systems 
achieve a higher rate than this in practise, but the level of 90% is used herein for 
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the sake of clarity. Thus, when the amount of statistics collected, starting from 
any given start point, such as a given time of a new day, for example 6 a.m v is 
sufficiently high that discrepancies from statistical variation are avoided, the 
system operator is able to set a threshold value of 10% failure rate. Thus, if on 
analysing the statistics, it is determined that more than 10% of attempted call 
set-ups have failed, then analysis with reference to the threshold indicates 
unacceptable performance and this is indicated to the system operator. In the 
present embodiment this analysis is carried out at OMC 130, and the indication 
is made by means of alarm indications computer screen information provided to 
human operators located at OMC 130. However, it is to be appreciated that in 
other examples such indication can be entirely automatic and can be directly 
interfaced with automatic equipment responses. 

However, when the collected statistics represent a smaD sample size, then 
statistical variations do have an effect, to the extent that a call set-up failure rate 
of more than the threshold of 10% can be determined by the statistical analysis 
even though in reality the system is operating healthily at a value lower than the 
threshold. For example, to exaggerate for the sake of understanding, consider 
the extreme case where the sample size was just 10 attempted call set-ups, in a 
system where on average call set-up failure rate was at an acceptable 5% level. 
Then, if by mere chance, i.e. effect of statistical variation, 2 of the 10 particular 
call set-ups failed, then according to the collected statistics the failure rate would 
be analysed as 20%, thus the threshold of 10% would be exceeded. To alleviate 
this effect, in the present embodiment the threshold is varied, more particularly 
the threshold is varied as a function of a sample size of said statistics derived 
from said communications system. In the present exaggerated example, a simple 
way of implementing this aspect of the invention is to set the failure rate 
threshold at the standard 10% level for sample sizes over, say, 1000 call set-ups 
(range 2), and to set the threshold at say a 50% level for sample size between 10 
and 1000 (range 1). Thus for high sample sizes the same accuracy is achieved as 
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in known methods, but for sample sizes between 10 and 1000 analysis is capable 
to an extent of indicating catastrophic failure levels (here higher than 50%) 
whilst vastly alleviating any false-alarm effect from statistical variation. 

It can be seen from FIG. 3, which illustrates the threshold 310 as a function of the 
two ranges described above, that the form in which the threshold is varied as a 
function of the sample size is essentially a stepped function, in which within 
each range the threshold is of the same function, namely constant, albeit it at 
different absolute levels. 

Alternatively, said sample size is divided into a plurality of ranges and said 
function of a sample size comprises respective functions for each respective 
range. In this case within each range there is a respective function, which 
functions are not all necessarily merely constant. In one version of the present 
embodiment, one or more of the respective functions are comprised of a linear 
equation. This is shown schematically in FIG. 4, where it can be seen that the 
threshold 410 in the first range, i.e. sample size between 10 and 1000 is 
comprised of a linear equation of the form y = mx + c, where y is threshold, x is 
sample size, m is gradient and c is the y intercept, and the threshold 420 in the 
second range, sample size of 1000 and beyond, is constant. (To avoid confusion, 
it is also noted that the sample size axis in FIGS. 3 and 4 is linear, i.e. it is not to 
be considered a logarithmic scale, rather it is merely coincidence that the two 
values shown are multiples of 10.) By virtue of the linear equation function, the 
trade-off and balance between being able to make use of smaller sample sizes 
whilst optimising accuracy of interpretation is further improved and enhanced. 

It is to be appreciated that in other versions of the present embodiment, the 
linear equation can be replaced by other functions, e.g. logarithmic functions or 
exponential functions, according to the requirements of the particular system 
under consideration. 
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The above values and functions have been shown in the form of an exaggerated 
example. Clearly, in practice, the different values, functions and ranges will be 
chosen by the skilled person according to the requirements of the particular 
system under consideration, and will accordingly give advantages even when 
variations introduced into the threshold are proportionally less dramatic than in 
the above example. Also, in practise the number of ranges in the plurality of 
ranges will often be more than the two described above. 

Also, in some cases the threshold under consideration will be one defined in 
terms of success rate, in which case the gradients or steps in the threshold 
function will be inverted compared to the above examples. 

In the above examples, sample sizes below 10 are not included in the analysis. 
However, if the desired trade-off is one in which as low a sample size as possible 
was required to be analysed, despite increased false-alarm or discrepancy rates, 
then range 1 as described above can be extended down to a sample size of 1. In 
this case although a certain level of false indications may still be expected, this 
level will still be reduced by the variable threshold of the present invention 
compared to if the threshold was not varied. Also, there will be situations where 
the statistics under consideration relates to data which lends itself to high 
accuracy threshold assessment under the present invention, even for, say, one 
sample. One such situation is for example where the statistics are derived from 
data related to relative transmission strength from either a BTS or a mobile 
station, where an individual value is on a continuous scale itself, although 
improved statistical accuracy is achieved at high sample sizes by consideration 
of average values. 

In one version of the presently described first embodiment, the threshold is 
varied by means of an operator system from which said function of a sample 
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size and values corresponding to said function can be ascertained by an operator 
using said operator system. One way of implementing this is for an operator 
system to be provided at OMC 130 for use by an operator. In addition to the 
operator system containing the technical means for varying the threshold, the 

5 operator system is also provided with a graphical user interface (GUI), which 
provides a display content which is schematically along the lines of the 
drawings shown in FIGS. 3 and 4. By clicking via a computer mouse anywhere 
on the threshold 310, 410 or 420, the operator is provided on the display with 
the values of the applicable threshold and the sample size corresponding to the 

1 0 point on the threshold at which he has clicked the mouse. Also, in the case of 
ongoing real time monitoring, the GUI can be arranged such that the applicable 
sample size and threshold at that moment is automatically displayed, along 
with any information about the results being achieved relative to the threshold. 
Specification of the details of the threshold variation implemented by the 

1 5 operator system can be either pre-programmed, programmable by the operator, 
or a combination of pre-programmed and programmable. Furthermore, by 
means of the GUI, the operator can select or amend factors such as the points at 
which the respective ranges start and finish, and so on. One way of doing this 
would be by dragging and dropping using a computer mouse. 

20 

A second embodiment of the present invention will now be described. The 
method of monitoring the communications system using statistics derived from 
said communications system according to the second embodiment is as shown 
in process flow chart 500 of FIG. 5, and comprises the step of collecting statistics 

25 from said communications system as shown in function box 510 of FIG. 5, and 
the step of statistically analysing said statistics with reference to a threshold, as 
shown at function box 520 of FIG. 5. Also, in the second embodiment the 
statistics again comprise the rate of success of call set-up in cells 152, 162, 172 
and 182 of communications system 100 as shown in FIG. 1, and the threshold 

30 again comprises an acceptable level of failure rate of call set-up. 
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In the present embodiment let us assume cells 152 and 162 represent busy 
urban cells, whereas cells 172 and 182 represent less busy rural cells. Under the 
present invention it is advantageous to vary the threshold such that a call set-up 
failure rate of, say, 10% is acceptable in the rural cells 172 and 182, whereas in 
the busy urban areas a better service is required, hence only a failure rate of 5% 
or less is acceptable in the urban cells 152 and 162. In this respect the second 
embodiment further includes the step of respectively allocating cells of said 
cellular communications system into a plurality of groups according to values of 
characteristics of said cells, as shown at function box 530 of FIG. 5. In the present 
case, urban cells 152 and 162 are allocated into a first group, called the urban 
group, and rural cells 172 and 182 are allocated into a second group called the 
rural group, i.e. the number of groups in the plurality of groups is in this 
particular example is two. The characteristic employed in making this allocation 
are, in the present case, a traffic channel usage level which is derived from 
stored data from the relevant cells. Other characteristics can be used instead or 
in addition. For example, in the present case where the groups are formulated 
according to the concept of rural or urban, further refinement can be introduced 
by including a characteristic related to the time profile of calls recorded from 
respective cells, e.g. day-time compared to evening and night time, also average 
call duration or average calling distance. Another characteristic that can be 
included or used instead is whether a cell is a macro-cell or a micro-cell. Yet 
another characteristic that can be included or used instead is details of the 
equipment type at that cell, for example it may be advantageous to use a 
different threshold for updated equipment compared to older equipment. On 
the basis of such grouping into rural cells and urban cells, in the second 
embodiment variation of said threshold comprises employing a respective 
group threshold value for each respective one of said plurality of groups, as 
already explained above where a threshold of 5% is applied to the urban cells 
and a threshold of 10% is applied to the rural cells. 
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The above described second embodiment is in the form of a relatively simple 
scenario. Clearly, in practice, the different characteristics, and form of 
groupings, will be chosen by the skilled person according to the requirements of 
5 the particular system under consideration. Also, in practise the number of 
groups in the plurality of groups will often be more than the two described 
above, and such larger numbers will provide for more refined application of the 
present invention. Another example of groupings is one with four groups, titled 
respectively "quiet", "normal", "busy", and "micro". 

10 

Also, although in the description above only four cells are included, normally a 
cellular communications system will include many more such cells. Also, the 
geographical boundaries of such cells, and other details such as frequency 
settings, power levels, and so on may often be changed in their nature due to the 

15 insertion of further BTSs in the system thus forming new cells. Thus the setting- 
up of a monitoring plan involving different thresholds for different cells is very 
time consuming and prone to errors. This further disadvantage is alleviated 
when, according to an optional aspect of the second embodiment, said cells are 
allocated into said plurality of groups automatically, and said values of 

20 characteristics of said cells comprise statistics collected automatically from said 
cellular communications system. In the present example the traffic channel 
usage data, constituting the characteristics of the cells, are collected 
automatically by BSC 140 and fed back from BSC 140 to MSC 110, where they 
are further fed to OMC 130. The means for automatically allocating the cells into 

25 the described plurality of groups can consist of standard processing equipment, 
including computer equipment. Such processing means, including any protocols 
or algorithms, are provided by the skilled person according to the requirements 
of the system under consideration, and in the usual manner the skilled person 
will also take into account previous experience gained in the system under 

30 consideration, or in other similar systems, or in simulations or trials or the like. 
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Also, the above described automatic allocation into groups can be augmented, 
amended, verified or fine-tuned by additional individual allocation of cells into 
the groups, under some degree of direct operator influence. One example is 
where individual cells represent special situations, such as a cell serving in- 
building coverage for a stock exchange or for a VIP. 

It is to be appreciated that in both the first and second embodiments above, the 
particular communications system components, the aspect monitored, and the 
particular thresholds considered are merely exemplary. The present invention is 
applicable to any aspects of a communications system in which performance 
monitoring is carried out with reference to thresholds. 

It is also to be appreciated that the specific system configuration shown in FIG. 1 
with reference to each of the above embodiments is merely exemplary, and the 
present invention is applicable to other system and network configurations. For 
example, OMC 130 may be coupled directly to BSC 140, instead of or in addition 
to being directly coupled to MSG 110, and may additionally be coupled directly 
to other BSCs not shown. In such cases the above described statistics may be 
transmitted direct from the respective BSC to the OMC. 

In the above described first and second embodiments, the means for carrying 
out the various method steps were described as being at or in OMC 130. It is 
however to be appreciated mat such means can be located in other parts of the 
communications system, and moreover may all be together or may alternatively 
be distributed at more than one location. The present invention can also be 
implemented by software or by hardware or by a combination of software and 
hardware. It is also to be appreciated that component parts of the means to 
implement the present invention can constitute dedicated equipment or can 
alternatively consist of existing equipment arranged to perform the herein 
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described method in addition to still also performing other functions, which 
other functions may in themselves already be known in the art. When in the 
form of dedicated equipment, such equipment may be located within existing 
communications systems parts, such as one or more OMCs, or may alternatively 
be located remotely in the form of specific dedicated equipment and coupled to 
the communications system, for example coupled to one or more OMCs. 
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CLAIMS 

1. A method of monitoring a communications system using statistics derived 
from said communications system; 

the method comprising the steps of: 
collecting statistics from said communications system; and 
statistically analysing said statistics with reference to a threshold; 
wherein said threshold is varied. 

2. A method as claimed in claim 1, wherein said communications system is a 
cellular communications system. 

J. A method according to claim 1 or 2, wherein said threshold is varied as a 
function of a sample size of said statistics derived from said 
communications system. 

t. A method according to claim 3, wherein said sample size is divided into a 
plurality of ranges and said function of a sample size comprises respective 
functions for each respective range. 

i- A method according to claim 3 or 4, wherein one or more of said functions 
are comprised of a linear equation. 

>. A method according to any of claims 2-5, wherein said threshold is varied 
by means of an operator system from which said function of a sample size 
and values corresponding to said function can be ascertained by an 
operator using said operator system. 

A method according to claim 2, further comprising the step of respectively 
allocating cells of said cellular communications system into a plurality of 
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groups according to values of characteristics of said cells; and wherein 
variation of said threshold comprises employing a respective group 
threshold value for each respective one of said plurality of groups. 

8. A method according to claim 7, wherein said cells are allocated into said 
plurality of groups automatically and wherein said values of 
characteristics of said cells comprise statistics collected automatically from 
said cellular communications system. 

9. A method according to claims 7 or 8, wherein said characteristics include a 
traffic channel usage level. 

10. A method according to any preceding claim wherein said statistics 
comprise a rate of success of call set-up. 

1 1 . An apparatus for monitoring a communications system using statistics 
derived from said communications system; 

the apparatus comprising: 

means for collecting statistics from said communications system; and 
means for statistically analysing said statistics with reference to a 
threshold; 

wherein said threshold is varied. 

12. An apparatus as claimed in claim 11, wherein said communications 
system is a cellular communications system. 

13. An apparatus according to claim 11 or 12, wherein said threshold is varied 
as a function of a sample size of said statistics derived from said 
communications system. 
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14. An apparatus according to claim 13, wherein said sample size is divided 
into a plurality of ranges and said function of a sample size comprises 
respective functions for each respective range. 

15. An apparatus according to claim 13 or 14, wherein one or more of said 
functions are comprised of a linear equation. 

16. An apparatus according to any of claims 13-15, wherein said threshold is 
varied by means of an operator system from which said function of a 
sample size and values corresponding to said function can be ascertained 
by an operator using said operator system. 

17. An apparatus according to claim 12, further comprising the step of 
respectively allocating cells of said cellular communications system into a 
plurality of groups according to values of characteristics of said cells; and 
wherein variation of said threshold comprises employing a respective 
group threshold value for each respective one of said plurality of groups. 

18. An apparatus according to claim 17, wherein said cells are allocated into 
said plurality of groups automatically and wherein said values of 
characteristics of said cells comprise statistics collected automatically from 
said cellular communications system. 

19. An apparatus according to claims 17 or 18, wherein said characteristics 
include a traffic channel usage level. 

20. An apparatus according to any of claims 1 1-19, wherein said statistics 
comprise a rate of success of call set-up. 



